You can only delete a replication set if it is not currently connected. If the replication set is connected, you must disconnect the connection and then delete the replication set.
To delete a replication set, right-click the replication set icon and select Delete. Additionally, you can highlight the replication set and press the Delete key on the keyboard.